Opening The Black-Box Of Referee Behaviour. An Agent-Based Model Of Peer Review
نویسندگان
چکیده
This paper investigates the impact of referee behaviour on the quality and efficiency of peer review. We focused especially on the importance of reciprocity motives to ensure cooperation between everyone involved. We modelled peer review as a process based on knowledge asymmetries and subject to evaluation bias. We built various simulation scenarios where we tested interaction conditions and manipulated author and referee behaviour. We found that reciprocity per se can have a negative effect on peer review as it tends to increase evaluation bias. It can have a positive impact only when purged by self-interest motivation and accompanied by disinterestedness and fairness standard.
منابع مشابه
Opening the Black-Box of Peer Review
This paper investigates the impact of referee behaviour on the quality and efficiency of peer review. We focused on the importance of reciprocity motives in ensuring cooperation between all involved parties. We modelled peer review as a process based on knowledge asymmetries and subject to evaluation bias. We built various simulation scenarios in which we tested different interaction conditions...
متن کاملDo Editors Have A Silver Bullet? An Agent-Based Model Of Peer Review
This paper presents an agent-based model of peer review that looks at the effect of different editorial policies of referee selection. We tested four author/referee matching scenarios as follows: random selection of referees, selection of referees with a similar status to submission authors, selection of higher-skilled and lower skilled referees. We tested these scenarios against three types of...
متن کاملCan Editors Compensate for the “Luck of the Reviewer Draw” Effect in Peer Review? An Agent-Based Model
This paper investigates the impact of various editorial options of author/referee matching and referee behaviour for the quality and efficiency of peer review. We modelled the peer review process as an interaction of editors, authors and referees characterized by intrinsic knowledge asymmetry and subject to evaluation bias. We built various scenarios where referees could behave randomly or foll...
متن کاملWhen Competition Is Pushed Too Hard. An Agent-Based Model Of Strategic Behaviour Of Referees In Peer Review
This paper examines the impact of strategic behaviour of referees on the quality and efficiency of peer review. We modelled peer review as a process based on knowledge asymmetry and subject to evaluation bias. We built two simulation scenarios to investigate largescale implications of referee behaviour and judgment bias. The first one was inspired by “the luck of the reviewer draw” idea. In thi...
متن کاملSaint Matthew strikes again: An agent-based model of peer review and the scientific community structure
This paper investigates the impact of referee reliability on the quality and efficiency of peer review. We modeled peer review as a process based on knowledge asymmetries and subject to evaluation bias. We tested various levels of referee reliability and different mechanisms of reviewing effort distribution among agents. We also tested different scientific community structures (cohesive vs. par...
متن کامل